feat: rewrite SKILL.md for MCP and improve skill-bench tests#65
Merged
feat: rewrite SKILL.md for MCP and improve skill-bench tests#65
Conversation
Install skill-bench via the official setup script if not already present. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Convert test cases from legacy TOML format (runner.sh + check scripts) to the skill-bench native format with built-in assertions. - Remove runner.sh and tools/ directory (no longer needed) - Convert all 5 test cases to new TOML format with [[checks]] - Update CLAUDE.md documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sonesuke
changed the title
- Use inline tables for command field in all test cases - Fix `param` → `tool-param` check command name - Add all available check types to AGENTS.md - Add .skill-bench/ to gitignore - Add skill-test task to mise.toml Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove cd from skill-test, run from project root - Add --plugin-dir for MCP plugin discovery - Move test cases from agents/skill-bench/cases/ to tests/ - Add skill-test to Tools table in AGENTS.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ults - Rewrite SKILL.md files to describe MCP tool calls with cypher patterns, following the google-patent-cli plugin pattern - Fix node labels in SKILL.md: (p:Paper) → (p:papers), (p:Paper) → (p:results) - Handle empty search results gracefully instead of crashing CypherEngine - Add execute_cypher checks to all functional tests - Add test cases for category, date, combined filters, and raw PDF download - Add cargo install to skill-test mise task Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sonesuke
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
(p:Paper)→(p:papers),(p:Paper)→(p:results)which caused cypher queries to return zero results{"count": 0}instead of crashing CypherEngine with "No suitable array found in JSON"execute_cypherinvocation checks to all functional tests to verify Claude uses cypher for result retrieval (not raw JSON reads)Test plan
mise run pre-commitpasses (fmt, clippy, test)mise run skill-test— 9/9 tests passpapers/results)🤖 Generated with Claude Code